Acoustic fingerprinting

ABSTRACT

Systems, methods, and other embodiments associated with acoustic fingerprint identification of devices are described. In one embodiment, a method includes generating a target acoustic fingerprint from acoustic output of a target device. A similarity metric is generated that quantifies similarity of the target acoustic fingerprint to a reference acoustic fingerprint of a reference device. The similarity metric is compared to a threshold. In response to a first comparison result of the comparing of the similarity metric to the threshold, the target device is indicated to match the reference device. In response to a second comparison result of the comparing of the similarity metric to the threshold, it is indicated that the target device does not match the reference device.

BACKGROUND

Criminal organizations user small, fast boats, high-speed helicopters,and small airplanes to smuggle illicit cargo or persons into sovereignterritories. It is not practicable to have human personnel visuallymonitoring thousands of miles of coastline or other territorialboundaries. The boats, helicopters, airplanes, and other vehiclesinvolved in smuggling may be tracked by conventional radar surveillance.But, radar surveillance risks alerting the smuggler, who may change hisbehavior and prevent or avoid interdiction operations. Further, radarsurveillance is unable to determine vehicle type or specificallyidentify individual vehicles.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate various systems, methods, andother embodiments of the disclosure. It will be appreciated that theillustrated element boundaries (e.g., boxes, groups of boxes, or othershapes) in the figures represent one embodiment of the boundaries. Insome embodiments one element may be implemented as multiple elements orthat multiple elements may be implemented as one element. In someembodiments, an element shown as an internal component of anotherelement may be implemented as an external component and vice versa.Furthermore, elements may not be drawn to scale.

FIG. 1 illustrates one embodiment of an acoustic fingerprinting systemassociated with acoustic fingerprint identification of devices.

FIG. 2 illustrates one embodiment of an acoustic fingerprinting methodassociated with acoustic identification of a target device.

FIG. 3 illustrates a waveform plot against three axes of examplemeasurements obtained from acoustic sensors monitoring a target device.

FIG. 4 illustrates one embodiment of an acoustic fingerprinting methodassociated with acoustic identification of a target device.

FIG. 5 illustrates an example three-dimensional bar plot of thesimilarity metric value between different devices.

FIG. 6 illustrates an embodiment of a computing system configured withthe example systems and/or methods disclosed.

DETAILED DESCRIPTION

Systems, methods, and other embodiments are described herein thatprovide acoustic fingerprinting of a device to automatically identifythe device based on sounds made by the device. In one embodiment, anacoustic fingerprinting system automatically extracts the most salientfrequencies from the sounds made by the device to form an acousticfingerprint, and determines how similar the acoustic fingerprint is to areference. In one embodiment, based on the similarity, the acousticfingerprinting system can state whether the device matches a known typeor even identify the device as a specific individual.

Such acoustic fingerprint identification is highly useful, for example,in surveillance of vehicles such as boats used for smuggling or otherillicit activities. For example, drug cartels and other criminalenterprises use small, fast “cigarette” boats, helicopters, and otheraircraft that stay below standard radar surveillance to deliver illegalcargos. Use of radar or lasers to track these vehicles has adisadvantage of being active surveillance involving directing detectableenergy at the target vehicle, alerting vehicle operators to thesurveillance. Advantageously, acoustic fingerprint surveillance ispassive, collecting sound wave vibrations emitted by operation of thetarget vehicle without the need to direct energy at the target. In oneembodiment, acoustic fingerprinting may identify these vehicles trackedby sound.

In one embodiment, the acoustic fingerprinting system generates anacoustic fingerprint for a target device from measurements of acoustic(sound) output of the target device. A similarity metric is thengenerated. The similarity metric quantifies similarity of the acousticfingerprint for the target device (also referred to as the targetacoustic fingerprint) to another acoustic fingerprint for a referencedevice (also referred to as the reference acoustic fingerprint). Thesimilarity metric is then compared to a threshold. In one embodiment,the threshold indicates a particular level of similarity between thetarget and reference acoustic fingerprints that distinguishes match andnon-match between the target and reference acoustic fingerprints, and,by extension, between the target and reference devices. In response to afirst result of the comparison of the similarity metric to thethreshold, the acoustic fingerprinting system indicates that the targetdevice matches the reference device. In response to a second result ofthe comparison of the similarity metric to the threshold, the acousticfingerprinting system indicates that the target device does not matchthe reference device.

It should be understood that no action or function described or claimedherein is performed by the human mind, and cannot be practicallyperformed in the human mind. An interpretation that any action orfunction described or claimed herein can be performed in the human mindis inconsistent with and contrary to this disclosure.

—Example Acoustic Fingerprinting System Overview—

FIG. 1 illustrates one embodiment of an acoustic fingerprinting system100. Acoustic fingerprinting system 100 includes an acoustic fingerprintgeneration component 105 configured to generate acoustic fingerprintsfrom input sound signals. Acoustic fingerprinting system 100 includes asimilarity metric generation component 110 configured to generate ametric that characterizes or quantifies a similarity between two or moreacoustic fingerprints. Acoustic fingerprinting system 100 includes amatch threshold comparison component 115 configured to determinewhether, based on a similarity metric, two acoustic fingerprints comefrom a same device, a device of similar type, make and/or configuration,or otherwise sufficiently similar device. Acoustic fingerprinting system100 includes a match/no match indication component 120 that isconfigured to signal results of the comparisons between acousticfingerprints. In one embodiment, acoustic fingerprinting system 100 mayinclude an acoustic fingerprint library 125 configured to store acousticfingerprints and associated data for subsequent reference. Each of theforegoing components and functions are described in further detailherein.

In one embodiment, acoustic fingerprint generation component 105 isconfigured to accept acoustic output of a device (such as acousticoutput 130 of a target device) as an input. In one embodiment, theacoustic output 130 of the target device is sound waveamplitude-vs-frequencies emitted by operation of a device. The acousticoutput is sensed by one or more acoustic transducers that convert theacoustic output to electrical signals representing the acoustic output.The electrical signals representing the acoustic output are provided asinput for acoustic fingerprint generation. Thus, in one embodiment, theacoustic output 130 of the target device may be provided as electricalsignals from one or more acoustic transducers detecting operation noisesof the target device. These electrical signals may be provided live, inreal-time, to acoustic fingerprint generation module 110 duringsurveillance of the target device. In one embodiment, these electricalsignals may be pre-recorded, and provided to acoustic fingerprintgeneration module 110 following surveillance of the target device.

In one embodiment, the acoustic output 130 of the target device formsthe basis of a target acoustic fingerprint 135 for the target device. Inone embodiment, the acoustic fingerprint is an acoustic operationsignature of a device, or a signature of the operational sounds of thedevice. Acoustic fingerprint generation module 105 may generate thetarget acoustic fingerprint 135 from the acoustic output 130. In oneembodiment, the target acoustic fingerprint includes values sampled overtime of selected frequencies within the broad-spectrum acoustic output130 of the target device. In one embodiment, where the target device isto be compared to a particular reference acoustic fingerprint, theselected frequencies are those selected for inclusion in the referenceacoustic fingerprint. The samples may be stored as time-series signals.In one embodiment, the target acoustic fingerprint includes a collectionof time-series signals of samples from selected frequencies of theacoustic output 130 of the target device.

In one embodiment, the target acoustic fingerprint 135 generated for thetarget device is provided as an input to the similarity metricgeneration component 110. In one embodiment, the target acousticfingerprint 135 is written to acoustic fingerprint library 125 forsubsequent use. In one embodiment, a reference acoustic fingerprint 140is also provided as an input to the similarity metric generationcomponent 110. In one embodiment, the reference acoustic fingerprint 140is generated by acoustic fingerprint generation module. In oneembodiment, the reference acoustic fingerprint 140 is retrieved fromacoustic fingerprint library 125, or from other storage locations.

In one embodiment, similarity metric generation component 110 generatesa similarity metric 145 that expresses the extent of similarity ordifference between the target acoustic fingerprint 135 and referenceacoustic fingerprint 140. In one embodiment, the similarity metricgeneration component 110 compares the target acoustic fingerprint 135 tothe reference acoustic fingerprint 140. The similarity metric generationcomponent 110 to determines an extent to which the target acousticfingerprint 135 and reference acoustic fingerprint 140 are similar,and/or an extent to which the target acoustic fingerprint 135 andreference acoustic fingerprint 140 differ. In one embodiment, similaritymetric 145 is a value that characterizes the extent of similarity ordifference.

In one embodiment, the value of the similarity metric 145 is compared toa threshold that indicates whether the target acoustic fingerprint 135and the reference acoustic fingerprint 140 are a match. In oneembodiment, the similarity metric 145 is provided as an input to matchthreshold comparison component 115. In one embodiment, a pre-determinedmatch threshold is provided as an input to match threshold comparisoncomponent 115. In one embodiment, the match threshold indicates a levelof similarity beyond which the target acoustic fingerprint and referenceacoustic fingerprint are considered to match each other. The thresholdmay vary based on whether the desired level of match is a lessrestrictive match of device type, such as make and model, or a uniquematch with a particular individual device. In one embodiment, the matchthreshold comparison component 115 produces a comparison result 150 thatindicates a match or no-match between the acoustic fingerprints.

In one embodiment, the comparison result 150 is provided to match/nomatch indication component 120. In one embodiment, match/no matchindication component 120 generates a signal indicating whether thetarget device is or is not a match to the reference device based on thecomparison result 150. This signal may be sent to other components orclient systems of acoustic fingerprinting system 100. In one embodiment,match/no match indication component 120 is a user interface, such as agraphical user interface configured to present an alert to a user ofacoustic fingerprinting system 100. In one embodiment, the alertindicates whether the target device is or is not a match to thereference device based on the comparison result 150.

Further details regarding acoustic fingerprinting system 100 arepresented herein. In one embodiment, the operation of the acousticfingerprinting system of FIG. 1 will be described with reference to anexample method shown in FIG. 2 . In one embodiment, generation of anacoustic fingerprint will be described with respect to an examplespectrogram shown in FIG. 3 . In one embodiment, identification of oneor more target devices by acoustic fingerprint will be described withrespect to an example method shown in FIG. 4 . In one embodiment,differentiation between multiple target devices by acoustic fingerprintwill be described with respect to an example similarity metric bar chartshown in FIG. 5 .

—Example Method for Acoustic Fingerprint Identification of a Device—

FIG. 2 illustrates one embodiment of an acoustic fingerprinting method200 associated with acoustic identification of a target device. At ahigh level, in one embodiment, acoustic fingerprinting method 200 is amethod for determining whether a target device matches a referencedevice based on similarity between acoustic fingerprints of the targetand reference devices.

As an overview, in one embodiment, the acoustic fingerprinting method200 generates an acoustic fingerprint for the target (also referred toas a target acoustic fingerprint). The target acoustic fingerprintprovides a signature of sounds made by the target device duringoperation. The acoustic fingerprinting method 200 then generates asimilarity metric that quantifies similarity between the target acousticfingerprint and a reference acoustic fingerprint. The reference acousticfingerprint provides a signature of sounds made by a reference deviceduring operation. The similarity metric provides a value that indicatesan extent to which the target and reference acoustic fingerprintsresemble each other or differ from each other. The acousticfingerprinting method 200 then compares the similarity metric to athreshold. The threshold may be a pre-determined threshold indicating anamount of similarity that distinguishes between a match or non-match ofthe fingerprints. By extension, match between the target and referenceacoustic fingerprints indicates match between the target and referencedevices. Accordingly, in response to the comparison result between thetarget and reference acoustic fingerprints, the acoustic fingerprintingmethod 200 indicates that the target device either matches, or does notmatch, the reference device.

In one embodiment, the acoustic fingerprinting method 200 initiates atstart block 205 in response to a processor (such as a processor ofacoustic fingerprinting system 100) determining one or more of: (i) acomputer configured as or as part of an acoustic fingerprinting system(such as system 100) has received or has begun receiving acoustic outputof a target device; (ii) acoustic surveillance of a target device hascommenced or has completed; (iii) a user (or administrator) of anacoustic fingerprinting system (such as system 100) has initiated method200; or (iv) that method 200 should commence in response to occurrenceof some other condition. Method 200 continues to process block 210.

At process block 210, the processor generates a target acousticfingerprint from acoustic output of a target device. In one embodiment,an acoustic fingerprint is a signature that characterizes sound producedduring the operation of a device. In one embodiment, this fingerprint orsignature includes acoustic information that is specific to theoperation of a specific device, such as acoustic components of enginenoise. Such an acoustic fingerprint may be used to uniquely identify adevice. Therefore, in one embodiment, the target acoustic fingerprint isgenerated from the acoustic output of the target device to be anacoustic operation signature that is specific to the target device.

In one embodiment, to generate the acoustic fingerprint, the processorperforms a bivariate frequency-domain to time-domain transformation ofthe fine frequency amplitude information in the acoustic output of thetarget device. The processor then forms the acoustic fingerprint fromtime series signals of amplitude values in selected frequency ranges.

In one embodiment, to effect the frequency-domain to time-domaintransformation, the processor divides or groups the fine frequencies ofthe acoustic output into coarse frequency bins. In one embodiment, therange of fine frequency signals in a coarse frequency bin arerepresented by a representative signal for the frequency bin.

The processor then selects a set of one or more of the frequency binsfor sequential sampling to create time series signals. In oneembodiment, where the acoustic fingerprint is an acoustic fingerprintfor a target device, the set of frequency bins selected for sampling arethose that are most salient—that is, most information bearing—about theoperation of a reference device to which the target device is to becompared.

In one embodiment, the processor creates a time series signal (TSS) foreach frequency bin in the set to create a TSS for the frequency bin. Inone embodiment, the processor creates a set of component TSSs bysampling representative frequencies of the set of bins to extract theiramplitude values at intervals over a period of time. In one embodiment,a sample is taken from a bin or representative frequency of the bin byretrieving the value of the representative frequency at a specifiedpoint in time, such as on the interval. In one embodiment, the processorsamples the amplitude of the representative signal of each frequency binat intervals to generate signal values of the TSS for the frequency bin.In one embodiment, the sampling rate for the TSS may be lower than thesampling rate of the representative frequency. The processor uses theTSSs as component signals of the acoustic fingerprint.

Process block 210 then completes, and method 200 continues at processblock 215. At the completion of process block 210, the processor hasgenerated a target acoustic fingerprint that describes or characterizesoperational acoustic output of a target device. This target acousticfingerprint may be compared with a reference acoustic fingerprint toidentify the target device. Further details regarding generation of atarget acoustic fingerprint from acoustic output of a target device aredescribed elsewhere herein.

At process block 215, the processor generates a similarity metric thatquantifies similarity of the target acoustic fingerprint to a referenceacoustic fingerprint of a reference device. In this way, the targetacoustic fingerprint may be compared to the reference acousticfingerprint to determine how similar or different the acousticfingerprints are from one another. The similarity metric is a value thatquantifies the results of the comparison between target and referenceacoustic fingerprints. The similarity metric thus describes similarityof the acoustic fingerprints, and by extension, the similarity of thetarget and reference devices. The similarity metric may be used as abasis for determining whether or not the target and reference devicesmatch.

In one embodiment, the corresponding component TSSs of target andreference acoustic fingerprints are compared. In this comparison, signalvalues of the corresponding component TSSs are compared pairwise to findan absolute error between each pair. The mean of the absolute error forthe values of the corresponding component TSSs is then calculated tofind a mean absolute error (MAE) between the component TSSs. The MAEquantifies similarity or likeness of the target and reference acousticfingerprint in the frequency range represented by the correspondingcomponent TSSs.

In one embodiment, this process of finding the MAE is performed for morethan one pair of corresponding component TSSs between the target andreference acoustic fingerprints. For example, this process of findingthe MAE may be performed for each pair of corresponding component TSSsbetween the target and reference acoustic fingerprints. The processorthen finds a cumulative MAE (CMAE) between the target and referenceacoustic fingerprints by finding the sum of the MAEs. The CMAE combinesthe MAEs between corresponding component TSSs to produce a singlesimilarity metric that quantifies overall similarity or likeness of thetarget and reference acoustic fingerprints.

With the generation of the CMAE similarity metric, process block 215completes, and method 200 continues at decision block 220. At thecompletion of process block 215, the complex question of the extent towhich acoustic output of a target device resembles acoustic output of areference device has been characterized or quantified in a simplesimilarity metric. The values of the similarity metric for target andreference acoustic fingerprints may be used to determine whether thetarget and reference devices match.

At decision block 220, the processor compares the similarity metric to athreshold. In one embodiment, the threshold describes a level ofsimilarity between a target and reference acoustic fingerprint thatdistinguishes between a match and non-match. The threshold level maydiffer based on how similar the acoustic output of a devices should befor them to be considered matches. The threshold level may also begoverned or dictated by the nature of the match. For example, a match ofa target device to a make and model may have a relatively lessrestrictive threshold. Or, for example, a match of a target device to aparticular individual device may have a relatively more restrictivethreshold. In one embodiment, where a lower similarity metric valuerepresents greater similarity (such as may be the case where thesimilarity metric is the CMAE), a relatively smaller or lower thresholdis more restrictive than a relatively larger or higher threshold.

In one embodiment, the processor evaluates whether or not the value ofthe similarity metric satisfies the threshold. In one comparison result,the similarity metric satisfies the threshold. For example, the value ofthe CMAE between the target and reference acoustic fingerprints may beless than or equal to the threshold. In another comparison result, thesimilarity metric does not satisfy the threshold. For example, the valueof the CMAE between the target and reference acoustic fingerprints maybe greater than the threshold.

Once the processor has determined whether or not the similarity metricsatisfies the threshold, decision block 220 then completes. In responseto the first comparison result, method 200 continues at process block225. In response to the second comparison result, method 200 continuesat process block 230. At the completion of decision block 220, theprocessor has determined whether the target acoustic fingerprint issufficiently like the reference acoustic fingerprint to be considered amatch.

At process block 225, in response to a first comparison result (of thecomparing of the similarity metric to the threshold) where thesimilarity metric satisfies the threshold, the processor indicates thatthe target device matches the reference device. In one embodiment, theprocessor composes and sends an electronic message indicating that thetarget device is a match to the reference device. In one embodiment, theprocessor causes a graphical user interface to display informationindicating that the target device is a match to the reference device.Process block 225 then completes, and method 200 continues to END block235, where method 300 completes.

At process block 230, in response to a second comparison result (of thecomparing of the similarity metric to the threshold) where thesimilarity metric does not satisfy the threshold, the processorindicates that the target device does not the reference device. In oneembodiment, the processor composes and sends an electronic messageindicating that the target device is not a match to the referencedevice. In one embodiment, the processor causes a graphical userinterface to display information indicating that the target device doesnot match the reference device. Process block 225 then completes, andmethod 200 continues to END block 235, where method 300 completes.

In one embodiment, as discussed in further detail elsewhere herein, thetarget device is either found to be the reference device itself, orfound not to be the reference device, based on the comparison of thesimilarity metric and threshold. Thus, in one embodiment, in response toa comparison result that indicates finding the match, the processorindicates that the target device is the reference device. And, in oneembodiment, in response to a comparison result that indicates notfinding the match, the processor indicates that the target device is notthe reference device.

In one embodiment, as discussed in further detail elsewhere herein, thetarget device is either found to be of a same type as the referencedevice, or found not to be of a same type as the reference device, basedon the comparison of the similarity metric and threshold. Thus, in oneembodiment, in response to a comparison result that indicates findingthe match, the processor indicates that the target device is a same typeof device as the reference device. And, in one embodiment, in responseto a comparison result that indicates not finding the match, theprocessor indicates that the target device is a different type of devicefrom the reference device.

In one embodiment, as discussed in further detail elsewhere herein,acoustic fingerprints are generated for target devices or referencedevices. In one embodiment, to generate an acoustic fingerprint,acoustic output of a device is measured. In one embodiment, a spectrumof the measurements is decomposed into a set of frequencies. In oneembodiment, the set of frequencies is partitioned into bins coveringranges of the set of frequencies. A set of one or more bins is selectedto be a basis of the acoustic fingerprint. Representative frequencies ofthe set of bins are selected as component frequencies are sampled atintervals over a period of time to produce a set of component timeseries signals. Where the device is the target device, the targetacoustic fingerprint is generated from the set of component time seriessignals. Where the device is the reference device, the referenceacoustic fingerprint is generated from the set of component time seriessignals. In one embodiment, ambient noise in the component time seriessignals is compensated for based on values for the component time seriessignals predicted by a machine learning algorithm.

In one embodiment, as discussed in further detail elsewhere herein,similarity metrics are generated from differences between componentsignals of a target acoustic fingerprint and corresponding componentsignals in a reference acoustic fingerprint. In one embodiment, togenerate the similarity metric, the processor finds a mean absoluteerror between a target component signal of a target acoustic fingerprintand a corresponding reference component signal of the reference acousticfingerprint. In one embodiment, this mean absolute error detection maybe repeated for a set of one or more target component signals includedin the target acoustic fingerprint. Once the mean absolute errors arefound for the target component signals in the set, the processor finds asum of the mean absolute errors. In one embodiment, the similaritymetric is the sum of the mean absolute errors. The sum of the meanabsolute errors may be referred to as the cumulative mean absolute error(CMAE).

In one embodiment, as discussed in further detail elsewhere herein, thereference acoustic fingerprint is generated from measurements ofacoustic output of the reference device. In one embodiment, as discussedin further detail elsewhere herein, the reference acoustic fingerprintis retrieved from a library of one or more acoustic fingerprints. In oneembodiment, the reference acoustic fingerprint is stored in the libraryin association with information describing the reference device.

In one embodiment, as discussed in further detail elsewhere herein, theacoustic output of the target device is recorded passively, and recordedusing one or more acoustic transducers.

In one embodiment, as discussed in further detail elsewhere herein, thetarget device is identified from among a plurality of devices. Here,measurements of acoustic output of the target device includemeasurements from the plurality of devices.

In one embodiment, as discussed in further detail elsewhere herein, thetarget device is a vehicle. In one embodiment, as discussed in furtherdetail elsewhere herein, the vehicle is one of a watercraft or anaircraft.

—Example Acoustic Fingerprint Generation—

In one embodiment, the acoustic fingerprinting systems and methodsdescribed herein leverage acoustic resonance spectrometry (ARS) toidentify devices such as vehicles based on acoustic output of the deviceduring operation. In one embodiment, measured or observed broad-spectrumacoustic output of a device undergoes a novel bivariate frequency-domainto time-domain transformation that characterizes the acoustic output byseparating it into bins of frequency ranges. From this characterizedacoustic output, the top frequency representatives that bestcharacterize the operation signature of the device are automaticallyidentified and used to form an acoustic fingerprint of that device.Thus, in one embodiment, the acoustic fingerprinting systems and methodsdescribed herein transform an acoustic waveform (for example, from adirectional microphone) into multiple time-series signals included in anacoustic fingerprint. In one embodiment, this acoustic fingerprint mayallow signals from passive acoustic transducers or sensors (such asmicrophones) to be consumed and analyzed in real-time for rapid, early,and accurate device identification.

As used herein with respect to items being surveilled by the acousticfingerprinting system, a device is a mechanical system being operatedwith one or more engines or motors. During operation, the engines ormotors emit (or cause the device to emit) acoustic output. The acousticoutput may be dynamic or changing in intensity, or vary as to frequencyof repetition, based on speed of operation of the engine or motor.

In one embodiment, an acoustic fingerprint as described hereincharacterizes a frequency domain signature of operation of a device inthe time domain by autonomously extracting selected most salient orinformative time series signals from measured acoustic output of thedevice. In one embodiment, acoustic fingerprint generation component 105is configured to identify and select the most salient frequencies forsampling as time series signals. In one embodiment, the time seriessignals for the one or more selected frequencies are used as thecomponent frequencies of an acoustic fingerprint for the device.

Advantageously, in one embodiment, the acoustic fingerprints asdescribed herein are compact because they rely on relatively few timeseries signals to accurately characterize operation of the device.Further, in one embodiment, due to the selection of a few most salienttime series signals, the sampling rate of the time series signals may berelatively low with little loss of discriminative power of thefingerprint. In one embodiment, one or both of these characteristics ofthe acoustic fingerprint as shown and described herein enablelow-overhead compute cost for acoustically identifying devices. Thisimprovement is not due to application of brute force computing power.

In one embodiment, to generate an acoustic fingerprint from acousticoutput of a device, the acoustic output of a device is first detected,sensed, or measured. In one embodiment, this detection, sensing, ormeasurement may be performed by acoustic transducers and a spectrumanalyzer. FIG. 3 illustrates a waveform plot 300 against three axes ofexample measurements 305 obtained from acoustic sensors monitoring atarget device (in this example, a boat). The frequency is on the x-axis310, time on the y-axis 315, and the acoustic power amplitude on thez-axis 320. Example measurements 305 shows how the acoustic output (orresponse, measured in power on z-axis 320) at each frequency changesover time. In one embodiment, the frequency domain waveforms over thespectrum of frequencies are sampled at a high sampling rate, such as 20kHz.

The measurements cover a spectrum of frequencies. For example, themeasurements may cover a spectrum of frequencies detectable by theacoustic transducers. In one embodiment, as shown in plot 300, thespectrum of frequencies approximately covers the range of human hearing,for example from 20 Hz to 20,000 Hz. In one embodiment, the spectrum offrequencies may extend below the range of human hearing into infrasoundfrequencies. In one embodiment, the spectrum of frequencies may extendabove the range of human hearing into ultrasound frequencies.

In one embodiment, the acoustic output of a target device is analyzed bya spectrum analyzer. The spectrum analyzer takes the raw electronicsignals from acoustic transducers sensing the acoustic output andconverts the electronic signals into a computer-readable representationof the acoustic output. The acoustic output is presented to the acousticfingerprinting system in the frequency domain. The acoustic output isrepresented over a frequency spectrum at a fine frequency resolutionpre-selected for the spectrum analyzer.

In one embodiment, the acoustic output is in the frequency domain. Inone embodiment, the acoustic output is a continuous frequency waveformspectrum output by acoustic transducers. For example, the acousticoutput detected by the acoustic transducers may be a sequence sampled attime intervals of frequency-domain waveforms over the spectrum offrequencies. The acoustic fingerprinting system then effects afrequency-domain to time-domain transformation to turn the acousticwaveforms into time series signals of operational signatures to be usedas components to create an acoustic fingerprint.

As an initial step of the frequency-domain to time-domaintransformation, the spectrum of frequency-domain measurements over timeis then decomposed into a set of raw frequencies. In one embodiment, theset of frequencies includes a frequency at intervals along the frequencyspectrum. In one embodiment, the frequencies are at intervals of the raw(fine frequency resolution) output by the spectrum analyzer, or coarserintervals.

In one embodiment, as a next step of the frequency-domain to time-domaintransformation, the set of frequencies is then partitioned into binscovering ranges of the set of frequencies. Thus, in one embodiment, thespectrum is divided into discrete bins that do not overlap. For example,processor divides the frequency spectrum of the acoustic output intofrequency bins. In one embodiment, the frequency bins are contiguousranges of the frequency spectrum. In one embodiment, the bins are ofapproximately equal width, covering similar range intervals of thefrequency spectrum.

In one embodiment, a frequency bin is represented by a discrete signalof acoustic amplitude over time within the frequency range of the bin.This signal may be referred to as the representative signal for thefrequency bin. In one embodiment, the representative signal for thefrequency bin may be a single selected frequency within the bin. Forexample, the representative signal may be the frequency signal amongthose within the bin that has highest peaks or that has greatest changesin amplitude. In another example, the representative signal may be afrequency signal at a selected location within the frequency range ofthe frequency bin, such as at a mid-point of the range. In oneembodiment, the representative signal may be an aggregate signal. Forexample, the aggregate signal may be an average (mean or median) of finefrequency values across the frequency range of the frequency bin.

The frequency bins and their representative signals may be considered tobe “coarse” because multiple raw frequencies are included in a frequencybin. For example, in one embodiment, the frequency spectrum may bedivided into 100 bins (although higher or lower numbers of bins may beused). Thus, in an acoustic spectrum ranging from 20 Hz to 20,000 Hz, afrequency bin that is one one-hundredth (0.01) of the width of thespectrum is approximately 200 (199.8) Hz wide. Referring again to FIG. 3, the original acoustic waveform of example measurements 305 is shownbinned into 100 independent bins.

In a further step of the frequency-domain to time-domain transformation,a set of one or more bins is selected to be a basis of the acousticfingerprint. In one embodiment, the selection of these bins is performedautomatically by the acoustic fingerprinting system. In one embodiment,where the acoustic fingerprint being constructed is a referencefingerprint, the selected bins are the most salient, or those that carrythe most information about the operation of the reference device.

In one embodiment, the target acoustic fingerprint is specificallygenerated for the purpose of comparison with a specific referencefingerprint, and therefore includes samples of acoustic output of thetarget at frequency ranges also sampled for acoustic output of thereference device. In one embodiment, where the acoustic fingerprintbeing created is a target acoustic fingerprint, the selected bins arethose bins used to create the reference acoustic fingerprint that thetarget acoustic fingerprint is to be compared to. In this way, thetarget acoustic fingerprint includes content most salient for comparisonwith the reference acoustic fingerprint.

In one embodiment, the most salient representative signals may beautonomously extracted and ranked by the acoustic fingerprinting systembased on a power spectral density (PSD) analysis of the representativesignals. In one embodiment, a PSD curve is generated for eachrepresentative frequency. Peaks in the PSD curve are dominated byrepetitive or cyclic output, such as motor/engine or other drive-trainnoise of the device under acoustic surveillance. Thus, the motor,engine, drivetrain, or other cyclic noises made by operating the deviceappear as peaks in the PSD curves. Those representative frequencieshaving the highest peaks in the PSD curve thus carry the mostinformation—sounds produced by operation of the device—and are thereforethe most salient.

In one embodiment, the bins are ranked by peak height of the PSD curvesfor the representative frequency of the bins. In one embodiment, the setof bins whose representative frequencies have the highest PSD peaks areautomatically selected to be the basis of the acoustic fingerprint. Inone embodiment, the top N bins are selected. N component time seriessignals for inclusion in the acoustic fingerprint will be sampled fromthese top N bins. In this way, the processor may autonomously extractand rank the most salient acoustic time-series signals from a databaseof measurements spanning a wide band of acoustic frequencies.

In one embodiment, N is 20. In one embodiment, N is between 1 and 20,inclusive. While N may be greater than 20, there are diminishing returnsof device identification accuracy for increases in the number ofcomponent signals N in an acoustic fingerprint and associated increasesin compute costs. In one embodiment, a value of approximately 20 for anumber N of component signals in acoustic fingerprints strikes a goodbalance between identification accuracy and compute cost, withidentification accuracy exceeding 95%. In one embodiment, the number ofbins and resulting component signals may be adjusted to other valuesdepending on the frequency ranges supported by the acoustic transducerand associated amplifier and processing hardware.

As used herein, the term “time series signal” refers to a data structurein which a series of data points (such as observations or sampledvalues) are indexed in time order. Representative frequencies for theset of N bins are sampled at intervals over a period of time to producea set of component time series signals to be components of an acousticfingerprint. Where the device is the target device, the target acousticfingerprint is generated from the set of component time series signals.Where the device is the reference device, the reference acousticfingerprint is generated from the set of component time series signals.In one embodiment, the sampling interval is modest, for example, aninterval of one second. Experiments have demonstrated selecting N to be20 bins and reporting their-frequency dependent power metrics at amodest interval such as 1 second results in good identificationperformance at a modest compute cost.

In one embodiment, an acoustic fingerprint includes a set of N timeseries signals of values sampled at intervals derived from the Nselected salient frequencies. These time series signals may be referredto as component signals of the acoustic fingerprint. In one embodiment,there are N component signals in the acoustic fingerprint, each of whichis sampled from a different one of the selected bins (that is, sampledfrom the representative frequency of the bin). For example, in oneembodiment, an acoustic fingerprint is a data structure that includesthe N component signals. At this point, the acoustic fingerprint hasbeen generated, and may be used for comparisons.

Thus, in one embodiment, the reference acoustic fingerprint includes Ncomponent signals, and target acoustic fingerprint acoustic fingerprintincludes N component signals. Thus, in one embodiment, the referenceacoustic fingerprint and the target acoustic fingerprint have an equalnumber of component signals. In one embodiment, these signals correspondto each other. This correspondence is based on the sampled frequency forthe component time series signal. For example, a first reference signalof the N component signals of the reference acoustic fingerprint issampled from the reference acoustic output at a first frequency, and afirst target signal of the N component signals of the target acousticfingerprint is sampled from the target acoustic output also at the firstfrequency.

In one embodiment, the reference acoustic fingerprint (provided forgeneration of the similarity metric) is generated from measurements ofacoustic output of the reference device. For example, the referenceacoustic fingerprint may be created from live acoustic output data, forexample, prior to or concurrently with detection of acoustic output ofthe target device and creation of the target fingerprint. In oneembodiment, the reference device is of unknown configuration. A createdreference acoustic fingerprint may be stored in a library or database ofacoustic fingerprints for subsequent retrieval or use.

In one embodiment, a user may be presented with an option to selectwhether the acoustic fingerprinting system is to generate a referencefingerprint or generate a target fingerprint, and in response to inputselecting one option or the other, the acoustic fingerprinting systemwill execute the user-selected option.

In some situations, it is possible that the component time seriessignals contain superimposed ambient noise. Therefore, in oneembodiment, after the creation of the component time series signals bysampling the selected most salient bins, the acoustic fingerprintingsystem may perform an ambient compensation technique. The ambientcompensation technique detects and corrects for superimposed ambientnoise. In one embodiment, a first portion of the values of eachcomponent time series signal are designated a training portion. Thefirst portions are then used to train a multivariate machine learningalgorithm (such as the multivariate state estimation technique) topredict the values of the component time series signals. A secondportion of the values of each component time series signal aredesignated a surveillance portion. The trained multivariate ML algorithmconsumes the surveillance portions of the component time series andpredicts their values. In one embodiment, the predicted values for thecomponent time series are recorded as a de-noised component time series.In one embodiment, one or more of the de-noised component time seriesare included in the acoustic fingerprint in place of the originalcomponent time series. This makes the acoustic fingerprinting techniquemore robust in high-noise areas (such as ports or harbors). The ambientcompensation further reduces the chance of false-positive or falsenegative identifications by the acoustic fingerprinting system. In thisway, the acoustic fingerprinting system may compensate for ambient noisein the component time series signals based on values for the componenttime series signals predicted by a machine learning algorithm.

—Example Similarity and Matching of Target and Reference Devices—

As mentioned above, the processor generates a similarity metric thatquantifies similarity of the target acoustic fingerprint to a referenceacoustic fingerprint. In one embodiment, the acoustic fingerprintingsystem 100 is configured to conduct a comparison test between acousticfingerprints of devices in order to generate the similarity metric.

In one embodiment, the acoustic fingerprinting system utilizes theacoustic fingerprints output from the acoustic fingerprint generationcomponent 105 to conduct a comparison test between acoustic operationsignatures of devices. In one example procedure for the comparison,initially, one device is chosen as the reference device or “GoldenSystem” (GS), and another device is chosen as the target device or “UnitUnder Test” (UUT). The acoustic operation signatures of these devicesare represented by their respective acoustic fingerprints. The acousticfingerprint of the reference device (or reference acoustic fingerprint)is compared to the acoustic fingerprint of the target device (or targetacoustic fingerprint).

In one embodiment, to compare the reference device to the target device,the acoustic fingerprinting system calculates the Mean Absolute Error(MAE) in a sequential one to one fashion. In one embodiment, the firstcomponent signal in the reference acoustic fingerprint is compared tothe first component signal in the target acoustic fingerprint, and thesecond component signal in the reference acoustic fingerprint iscompared to the second component signal in the target acousticfingerprint, and so on through the correlated pairs of componentsignals. The resulting MAE values are then summed to distil thedifferences between the two signatures into a similarity metric calledthe Cumulative MAE (CMAE). In one embodiment, this process is repeatedfor any remaining target devices.

On a macro scale this process may assist in quantitativelydifferentiating between different models of devices allowing for apassive identification of an exact device under question. To accomplishthis one device (e.g., a boat) under surveillance would be chosen as thereference device and the remaining devices in a group (e.g., boats in afleet) would be chosen as reference devices and compared to thatreference device. The CMAEs for identical devices will be drop to zerowhile devices of different make and model will have large values andtherefore indicate a difference in make and model. The CMAE similaritymetric may therefore be used to, for a target device, identify themake/model of the target device from a library of acoustic fingerprintsstored for multiple makes/models of boats.

As discussed above, in one embodiment, an acoustic fingerprint includesa set of N component time series signals (TSS) of values sampled atintervals from selected salient frequencies. In one embodiment, thecomponent TSSs of acoustic fingerprints allow for comparison of oneacoustic fingerprint to another in the time domain. For example, bycomparing values of a component TSS in the acoustic fingerprint tovalues of a corresponding component TSS in another acoustic fingerprint,similarity (or difference) between the component TSSs may be quantified.This process of comparison to quantify similarity between correspondingTSSs in acoustic fingerprints may be repeated for remaining oradditional corresponding pairs of component TSSs to quantify overallsimilarity between the acoustic fingerprints.

In one embodiment, the processor compares the values of component timeseries signals in the target acoustic fingerprint (also referred toherein as a target component TSS) with the values of correspondingcomponent time series signals in the reference acoustic fingerprint(also referred to herein as a reference component TSS). In oneembodiment, the reference acoustic fingerprint has reference componentTSSs for a similar (or same) set of frequency bins as does the targetacoustic fingerprint. Thus, in one embodiment, a target component TSScorresponds to a reference component TSS where they are both sampledfrom the similar bins.

In one embodiment, values of the target component TSS are comparedpairwise with the values of the corresponding reference component TSS.In one embodiment, an initial pair of values are selected, one valuefrom the target component TSS and one from the reference component TSS.In one embodiment, the pair of values selected are the values occupyingbeginning (or end) positions of the target component TSS and referencecomponent TSS. In one embodiment, other locations in the component TSSsignals may be selected for comparison.

In one embodiment, the initial values are then compared to find anextent to which the values differ. For example, the values may becompared by finding an absolute value of the difference or residualbetween them. This absolute value of the difference may also be referredto as an absolute error between the paired values. Additional pairs ofvalues from the target and reference component TSSs are subsequentlycompared to find an absolute value of the difference between the pair.In one embodiment, each pair of values following the initial values ofthe component TSSs are compared in turn to find an absolute errorbetween each pair. In one embodiment, a subset of the pairs of values ofthe component TSS are compared to find an absolute error between eachpair in the subset. For example, some pairs may be skipped, for exampleby comparing only value pairs appearing at an interval in the componentTSSs.

In one embodiment, the processor calculates a mean of the absoluteerrors between the paired value of the corresponding target andreference component TSS to generate a mean absolute error (MAE) forthese corresponding component TSSs. In one embodiment, the processorcalculates a MAE between each corresponding pair of target and referencecomponent TSSs of the target and reference acoustic fingerprints. Then,in one embodiment, the processor calculates a cumulative MAE (CMAE)between the target and reference acoustic fingerprints from the set ofthe MAEs between the component signals. In one embodiment, the processorcalculates the CMAE by combining the MAEs, for example by adding up allthe MAEs to find the sum of the MAEs. In one embodiment, the CMAEbetween the target and reference acoustic fingerprints is used as asimilarity metric to quantify the similarity or likeness of the targetand reference acoustic fingerprints.

In one embodiment, other similarity metrics may be substituted for theCMAE. In one embodiment, other similarity metrics that quantifysimilarity in the time domain of the corresponding component TSSs fortarget and reference acoustic fingerprints may be acceptable alternativesimilarity metrics to CMAE. For example, similarity metrics betweentarget and reference acoustic fingerprints based on mean absolute scalederror, mean squared error, or root mean square error between thecorresponding target and reference component TSSs may also performacceptably.

In one embodiment, finding a match between target and reference acousticfingerprints indicates that the target device is of a same type as thereference device. For example, the target device may be of the same typeas the reference device where the target device has the same make andmodel as the reference device. The target device being the same type asthe reference device does not necessarily mean that the target device isthe identical unit as the reference device.

Note that stochastic differences between devices of the same make andmodel is enough to uniquely identify the device. Thus, in oneembodiment, finding a match between target and reference acousticfingerprints indicates that the target device is the reference device.In other words, the target device and reference device are a match whenthey are the same device.

—Passive Acoustic Surveillance—

In one embodiment, the acoustic output of the target device is recordedpassively, for example by one or more acoustic transducers. For example,in one embodiment, acoustic energy is not directed to the target deviceby the acoustic fingerprinting system. Instead, energy collected fromthe target device is generated by operation of the target device (suchas motor or engine noise) or generated by interaction of the targetdevice with its surroundings (such as sounds of a boat hull on water ortire noise on a road).

Advantageously, the passive nature of acoustic surveillance minimizesrisk of the surveillance being detected. For example, passive recordingof acoustic output does not alert operators of a target vehicle to thesurveillance. This is in contrast to active surveillance activities suchas RADAR, LIDAR, or SONAR, which respectively direct radio, laser, orsound energy towards the target vehicle. These active surveillanceactivities may be detected by operators of the target device, who maythen abort any illicit activity.

—Example Acoustic Transducers—

As used herein, an acoustic transducer refers to an apparatus thatconverts sound wave vibrations into electrical signals when exposed tothe sound wave vibrations. For example, an acoustic transducer may be amicrophone, hydrophone, or geophone as discussed in further detailherein. The electrical energy generated by the transducer from the soundwave vibrations may be amplified by an amplifier and/or recorded as adata structure in various media.

In one embodiment, the acoustic fingerprinting system includes one ormore acoustic transducers for sensing or recording acoustic output of adevice. In one embodiment, acoustic output of the target device isrecorded using one or more acoustic transducers. In one embodiment,acoustic output of the reference device is recorded using one or moreacoustic transducers. Differences between a set of acoustic transducersused to record acoustic output of a target device and a set oftransducers used to record acoustic output of a reference device may becorrected for by the acoustic fingerprinting system.

In one embodiment, the acoustic transducer may be a sphericallyisotropic transducer that receives sound wave vibrations from multipledirections. In one embodiment, the acoustic transducer may be adirectional transducer that collimates incoming sound wave vibrationsfrom a particular direction to the transducer through a shaped channel(such as through a round or rectangular tube). In one embodiment, theparticular direction is a direction toward a target device or referencedevice. In one embodiment, the acoustic transducer may be a directionaltransducer that concentrates incoming sound wave vibrations from aparticular direction to the transducer by reflecting the sound wavevibrations off of a reflecting inner surface such (such as off aparabolic surface or partial spherical surface). The concentratingdirectional transducer concentrates soundwaves impinging on a largeropening where the sound waves come in approximately parallel from atarget source. In one embodiment, a directional transducer serves toexclude ambient noise from the sensed acoustic output of a targetdevice. Transducers with varying sensitivity based on direction may alsobe used.

In one embodiment, the acoustic fingerprinting system uses a pluralityof (or multiple) transducers. For example, the plurality of transducersare independent directional microphones. The plurality of transducers isdeployed with at least several inches of separation between thetransducers. In one embodiment, the plurality of transducers includestwo concentrating directional microphones. Employing two or moremicrophones deployed with several or more inches of separation permitscontinuous triangulation. The triangulation allows the system toestimate with fairly high accuracy the location of a vehicle undersurveillance. This allows for more precise labeling of samples asbelonging to a particular vehicle under surveillance. The triangulationalso allows the system to infer a rate at which a vehicle is comingcloser or going away. This allows for compensation for Doppler shifts infrequency in the acoustic output received by the system.

In one embodiment, the acoustic fingerprinting system uses just onetransducer. Where just one transducer is used, the acousticfingerprinting system compensates for Doppler shifts in frequency bysending a pulsed signal and inferring bounce-back time.

In one embodiment, an acoustic transducer may be anelectromagnetic-acoustic transducer, such as a condenser transducer, adynamic transducer, or a ribbon transducer. In a capacitance orcondenser transducer, a diaphragm acts as one plate of a capacitor, inwhich the electrical signals are produced as electrical energy acrossthe capacitor plates is changed when the sound wave vibrations displacethe diaphragm. In a dynamic or moving-coil transducer, an induction coilis placed in a magnetic field, and the electrical signals are producedby induction as the induction coil is displaced within the magneticfield by the action of the sound wave vibrations (for example by actionon a diaphragm attached to the induction coil). In a ribbon transducer,a conductive ribbon is suspended in a magnetic field, and the electricalsignals are produced by induction as the ribbon is displaced within themagnetic field by the action of the sound wave vibrations.

In one example, the acoustic transducer may be a piezoelectric-acoustictransducer that generates electrical energy in proportion to the soundwave vibrations when a piezoelectric material is deformed by the soundwave vibrations. In one example, the acoustic transducer may be anoptical-acoustic transducer that converts sound wave vibrations intoelectrical energy by sensing changes in light intensity, such as in afiber-optic or laser microphone. Other acoustic transducers forgenerating electrical signals from sound wave vibrations may also beused in accordance with the acoustic fingerprinting systems and methodsdescribed herein.

—Acoustic Fingerprint Library—

In one embodiment, the reference acoustic fingerprint is retrieved froma library (or other data structure(s)) of acoustic fingerprints. Thereference acoustic fingerprint is stored in the library in associationwith information describing the reference device. In one embodiment, alibrary of acoustic fingerprints is maintained by one or more parties(including third parties), such as government entities or devicemanufacturers. For example, a government entity may acoustically surveilone or more vehicles of a given make and model, and generate (and add tothe library) an acoustic fingerprint for those vehicles. Thesesurveillance acoustic fingerprints may serve as reference fingerprintsfor the make and model of the surveilled device, as well as a uniquefingerprint of the surveilled device. Or, for example, where the deviceis a vehicle, legitimate vehicle manufacturers will generate and supplyto the library acoustic fingerprints for various makes and models of themanufacturer's vehicles.

Association between information in the library may includecomputer-readable relationship or connection between the acousticfingerprint data and the data describing the reference device, forexample, sharing a row in a table, referencing keys of other tables,linking between data values, or other affiliation of data. In oneembodiment, the information describing the reference device may includevalues that describe type, make, model, configuration, or other physicalproperties of the device. The information describing the referencedevice may include operation parameters of the reference device duringcollection of acoustic output to generate the reference fingerprint,such as throttle position or speed. The information describing thereference device may include location of the reference device duringcollection of acoustic output to generate the reference fingerprint, forexample by GPS coordinate, latitude and longitude, address, or othergeolocation information. The information describing the reference devicemay include a unique identifier of the device, such as a serial number,vehicle identification number, vehicle registration number, or otherdescriptor of a specific device or unit.

—Example Application of Acoustic Fingerprinting—

As mentioned above, acoustic fingerprinting for acoustic identificationof devices finds one application in vehicle surveillance andinterdiction. For example, the acoustic fingerprinting systems andmethods may be used to track and identify boats or other vehiclesinvolved in illegal cargo distribution. While example embodiments may bedescribed herein with respect to such vehicle surveillance andinterdiction, the acoustic fingerprinting systems and methods describedherein may be used to identify a broad range of devices.

In one embodiment, the acoustic fingerprinting systems and methodsdescribed herein identify or match vehicles. In one embodiment, theacoustic fingerprinting systems and methods identify a make and model ofvehicle. In one embodiment, the acoustic fingerprinting systems andmethods not only identify a make and model of a vehicle, butadditionally uniquely identify the exact vehicle.

In one embodiment, the target device is a vehicle. As used herein, avehicle is a self-propelled device for transporting persons or things. Avehicle may be, for example, a watercraft for transporting persons orthings on or in water, such as a boat, ship, submarine, submersible,personal watercraft or jet-ski, or hovercraft. A vehicle may also be,for example, an aircraft for transporting persons or things by air, suchas an airplane, helicopter, multi-copter (for example a quadcopter),autogyro or gyrocopter, ultralight, blimp, dirigible, or semi-rigidairship. A vehicle may also be, for example, a land craft fortransporting persons or things over land, such as an automobile, atruck, a locomotive or train, a tank or other armored vehicle. In oneembodiment, the target device is a watercraft or an aircraft. In oneembodiment, vehicles may be piloted or controlled by an operator onboard the vehicle. In one embodiment, vehicles may be remotely operatedor remote controlled by an operator away from the vehicle, such as in adrone aircraft. Vehicles may be autonomous or self-driving, where theoperator is computer logic. Vehicles may be non-autonomous, where theoperator is a person.

In one embodiment, the acoustic fingerprinting systems and methodsdescribed herein may be used for passive surveillance of boats or otherwatercraft. Surveillance equipment such as acoustic transducers may beplaced on shore for surveillance of watercraft in a harbor or port, orotherwise near a coastline. Surveillance equipment such as acoustictransducers may be placed on ships or otherwise in or on the water forsurveillance of watercraft at sea or offshore.

In one embodiment, groups of acoustic transducers deployed acrossstretches of coastline and/or areas of water may be interconnected inorder to provide multiple points of surveillance of watercraft. Theinterconnection may be by data networks between computing devices thatcollect acoustic information from the acoustic transducers. This allowswatercraft to be tracked as they move through a region.

In one embodiment, where an acoustic fingerprint for a target device isnot in the library of acoustic fingerprints, for example during aninitial surveillance or monitoring of the target device, the acousticfingerprint for the target device may be stored in the library as areference acoustic fingerprint. In one use case, where a targetfingerprint of a boat is not in the library of known acousticfingerprints for known makes, models, or individual devices, theacoustic fingerprint is stored as a reference and used to positivelyidentify the vessel when interdiction craft are able to stop the vessel.

FIG. 4 illustrates one embodiment of an acoustic fingerprinting method400 associated with acoustic identification of a target device. In oneembodiment, method 400 is one example of a CMAE-based comparisonalgorithm for differentiating devices. At a high level, a referenceacoustic fingerprint is generated from acoustic output of a referencedevice for one or more configurations of the reference device, and thenone or more target acoustic fingerprints are generated from acousticoutput of reference device(s) and compared with the reference acousticfingerprint(s).

In one embodiment, acoustic fingerprinting method 400 initiates at startblock 405 in response to a processor (such as a processor of acousticfingerprinting system 100) determining one or more of: (i) a computerconfigured as or as part of an acoustic fingerprinting system (such assystem 100) has received or has begun receiving acoustic output of areference device; (ii) acoustic surveillance of a reference device hascommenced or has completed; (iii) a user (or administrator) of anacoustic fingerprinting system (such as system 100) has initiated method400; or (iv) that method 400 should commence in response to occurrenceof some other condition. Method 400 continues to process block 410.

At process block 410, the processor initializes the reference device(also referred to as a golden system or GS) with a set of M totalallowable configurations. In one embodiment, the allowableconfigurations of the reference device include manufacturer variationsof a particular make and model of device. For example, where the deviceis a boat, allowable configurations may include different motors, suchas a 4-cylinder engine configuration, a 6-cylinder engine configuration,an 8-cylinder engine configuration, etc., with further variation forexample based on fuel type such a diesel engine configurations andgasoline engine configurations. In one embodiment, the allowableconfigurations include those variations that affect operating noise,such as drivetrain options, and may exclude those variations that do notaffect operating noise, such as paint color. In one embodiment,therefore, M may be a number of known variations in configuration of amake and model of reference device.

At process block 415, the processor initiates a counter i for an outerloop that repeats for each of the M configurations. The outer loop isheaded by decision block 420. At decision block 420, the processordetermines whether the counter i is less than or equal to the number ofconfigurations M. Where this condition is true (decision block 420:YES),the outer loop proceeds through an iteration, continuing at processblock 425.

At process block 425, the processor scans the reference device (GS) thathas been placed in a configuration Mt. In one embodiment, themeasurements of the acoustic output of the reference device in theparticular configuration are taken, for example as described in detailabove. Processing then continues to process block 430.

At process block 430, the processor extracts one hundred (100) frequencytime series from the measurements of the acoustic output of thereference device (GS) in configuration M_(i), for example as describedin detail above. In one embodiment, the acoustic measurements are thusconverted to a coarse set of bins (in this case, 100 bins). Processingthen continues to process block 435.

At process block 435, the processor determines twenty (20) frequencybins X_(i) ²⁰ and extracts out 20 time series GS_(i) ²⁰ for thereference device (GS) in configuration M_(i), for example as describedin detail above. In one embodiment, a smaller subset (in this case, 20bins) of the frequencies are identified to be most salient, that is, thefrequencies that are most useful and pertinent to inform about the make,model, type, or identity of the reference device. In one embodiment,time series signals are extracted from the 20 bins that were identifiedas most salient, for example by sampling them at intervals, as describedabove. These time series signals will be used as component signals ofacoustic fingerprints for the reference device (GS) in configuration Mt.Processing then continues to process block 440.

At process block 440, the processor creates a three-dimensionalfingerprints surface GS_(i) ^(3D) for the reference device (GS) inconfiguration Mt. In one embodiment, the acoustic fingerprint for thereference device (GS) in configuration M_(i) is created from thecomponent signals selected at process block 435. In one embodiment, theacoustic fingerprint for the reference device (GS) in configurationM_(i) is created as a three-dimensional surface in dimensions offrequency, time, and acoustic power amplitude. In one embodiment, thethree-dimensional fingerprint surface GS_(i) ^(3D) combines thecomponent signals, with the amplitude of each component signal extendingover the range of its bin on the frequency axis. Processing thencontinues to process block 445.

At process block 445, the processor proceeds with a number N of targetdevices (also referred to as units under test (UUT)). In one embodiment,the number of target devices (UUTs) is counted. Processing thencontinues to process block 450, where the processor initiates a counterj for an inner loop that repeats for each of the N target devices(UUTs). The inner loop is headed by decision block 455. At decisionblock 455, the processor determines whether the counter j is less thanor equal to the number of target devices (UUTs) N. Where this conditionis true (decision block 455:YES), the inner loop proceeds through aniteration, continuing at process block 460.

In the inner loop, the measurements of acoustic output are repeated forone or more target systems. The component time series signals aresampled from the acoustic output of the target system at the samefrequency bins determined for the reference system. This enablescomparison of target and reference component TSSs one-to-one, at thesame frequencies. With this one-to-one comparison between the selectedfrequencies, it becomes clear whether there is or is not a differencebetween a target component TSS and the reference component TSS.

At process block 460, the processor scans the target device UUT_(j). Inone embodiments, the measurements of the acoustic output of targetdevice UUT are taken, for example as described in detail above.Processing then continues to process block 465.

At process block 465, the processor extracts twenty (20) acoustic timeseries UUT_(j) ²⁰ utilizing the prior-determined twenty frequency binsX_(i) ²⁰ from the acoustic output of the target device UUT_(j). Theprior determined bins X_(i) ²⁰ are those bins or ranges of frequencydetermined or selected when the reference device (GS) in configurationM_(i) was scanned. In one embodiment, the processor extracts thecomponent TSS (UUT_(j) ²⁰) for the twenty bins by sampling therepresentative frequencies of these bins at intervals, for example asdescribed in detail above. Thus, in one embodiment, the bins X_(i) ²⁰are sampled from the acoustic output of the reference device to generatecomponent signals for the reference device fingerprint (for example asdescribed above with reference to process block 435), and then the binsX²⁰ are sampled again from acoustic output of the target device togenerate component signals for the target device fingerprint. Processingthen continues to process block 470.

At process block 470, the processor creates a three-dimensionalfingerprints surface UUT_(j) ^(3D) for the target device UUT_(j). In oneembodiment, the acoustic fingerprint for the target device is createdfrom the component signals extracted at process block 465. When plottedtogether in dimensions of frequency, time, and acoustic power amplitude,the component signals of the acoustic fingerprint for the target deviceUUT_(j) form a three-dimensional fingerprints surface UUT_(j) ^(3D).Processing then continues to process block 475.

At process block 475, the processor computes three-dimensional residualsR_(i-j) between component time series signals GS_(i) ²⁰ for thereference acoustic fingerprint and component time series signals UUT_(j)²⁰ for the target acoustic fingerprint. In one embodiment, the processorcompares the corresponding time series signals to find the difference orresiduals between the corresponding pairs of target and referencecomponent signals, for example as described in detail above. When thecomputing of residuals is repeated for each pair of target and referencecomponent signals, the three-dimensional residuals R_(i-j) are computedbetween UUT_(j) ^(3D) and GS_(i) ^(3D). Processing then continues toprocess block 480.

At process block 480, the processor performs a mean absolute error (MAE)computation to produce mean absolute errors MAE_(i-j) on thethree-dimensional residuals R_(i-j). In one embodiment, the MAE valuesbetween the corresponding reference and target component time seriessignals for each frequency are determined, for example as described indetail above. In this example, where there are 20 frequencies, theprocessor finds the MAE value between the corresponding component timeseries signals for each of those bins, resulting in 20 MAE valuesMAE_(i-j). Processing then continues to process block 485.

At process block 485, the processor sums the mean absolute errorsMAE_(i-j) to compute the cumulative mean absolute error (CMAE), andstores the CMAE for subsequent use. In one embodiment, the CMAE is usedas a similarity metric to determine whether or not target device UUT_(j)is a match to reference device GS in configuration M_(i), for example asdescribed in further detail above. Processing then continues to processblock 490.

At process block 490, counter j for the inner loop is incremented j=j+1,and processing returns to decision block 455 to determine whether or notthe inner loop is to be repeated for another iteration, for anothertarget device. In response to the determination at decision block 455that the inner loop is not to be repeated for another iteration(decision block 455: NO), processing continues to process block 492,where counter i for the outer loop is incremented i=i+1. Processing thenreturns to decision block 420 to determine whether or not the outer loopis to be repeated for another iteration, for another configuration ofthe reference device. In response to the determination at decision block420 that the outer loop is not to be repeated for another iteration(decision block 420: NO), processing continues to END block 495, wheremethod 400 completes.

At the conclusion of method 400, CMAE similarity metrics have beencreated for each comparison of the M configurations of the referencedevice with the N target devices. These similarity metrics may becompared to a threshold, as described in detail above to determinewhether any of the N target devices match any of the M configurations ofthe reference device.

—Multiple Device Identification—

In one embodiment, the target device is one among a plurality ofdevices. In one embodiment, each of the plurality of devices may beacoustically surveilled as additional target devices with one or moreacoustic transducers. Thus, in one embodiment, the processor maypassively record acoustic output of the plurality of devices includingthe target device using the one or more acoustic transducers. Theprocessor may then identify the target device from among the pluralityof devices. The identification may be based on comparison of theacoustic fingerprint of the target device with acoustic fingerprints ofthe other devices.

In one embodiment, the acoustic output for devices in the plurality ofdevices is cross-compared with the acoustic output of other devices inthe plurality of devices by similarity metric in order to differentiatethe devices. In one embodiment, the devices are differentiated by type(such as by make and model). In one embodiment, the devices aredifferentiated by identity.

FIG. 5 illustrates an example three-dimensional bar plot 500 of thesimilarity metric value between different devices. While the plot 500 isshown and described with respect to an example use case of acousticallydifferentiating boats under surveillance by make and model, theprocedures for acoustic differentiation may be more generally applied toacoustically differentiating a plurality of devices.

In one embodiment, plot 500 shows the CMAE similarity metric valuesbetween each member of a set of four vessels under surveillance plottedagainst three axes: a reference device axis, x-axis 505, a target deviceaxis, y-axis 510, and a CMAE similarity metric axis, z-axis 515. TheCMAE value between an acoustic fingerprint of a reference device and anacoustic fingerprint of a target device is plotted as a bar at theintersection of the reference device and the target device in the x-yplane of plot 500.

In one embodiment, plot 500 illustrates output from the acousticfingerprint identification process as applied to multiple devices, asapplied in an example use case of differentiating makes and models ofboats under acoustic surveillance. In one embodiment, the output fromthe acoustic fingerprint identification process may be presented in agraphical user interface that displays a bar plot of CMAE similaritymetrics between reference and target devices, such as plot 500.

In one embodiment, acoustic output has been collected for each of fourunique vessels of differing make and model. Reference and targetacoustic fingerprints have been generated for each device, for exampleas shown and described above with reference to FIGS. 2 and 4 . In oneembodiment, both the reference and target acoustic fingerprints for adevice are calculated from the same acoustic output measurements. CMAEsimilarity metric values between the four metrics between the fourdevices have been calculated, stored, and retrieved for display in plot500. In one embodiment, the CMAE similarity metrics are normalized tocorrect for differences in acoustic energy or loudness of themeasurements acoustic output from the devices.

The first row 520 indicates the CMAE similarity metric values of vessel#1 as it is leaving port compared to itself, vessel #1 525, and to thethree remaining boats under surveillance: vessel #2 530, vessel #3 535,and vessel #4 540. The first CMAE similarity metric value for vessel #1525 is zero, because the measurements from vessel #1 are compared toitself. Comparing vessel #1 to other makes and models, the CMAEsimilarity metric is much higher. For example, as seen at reference 530,the CMAE similarity metric value for comparing vessel #1 with vessel #2is approximately 2.0 dB. The higher similarity metric value than occurswith the self-comparison indicates that the vessels are not identical.This provides evidence that any further surveillance can be confidentlytracked to a unique boat. In one embodiment, this process is repeatedwith all combinations of boats under surveillance, as seen in theremaining target device rows of plot 500.

—Selected Advantages—

In one embodiment, the acoustic fingerprinting systems and methodsdescribed herein can identify an exact make/model of a device such as avehicle. In one embodiment, the acoustic fingerprinting systems andmethods described herein can identify an exact individual device. Forexample, in an interdiction use case, where ten identical make/modelboats are being watched in a port, and only boat #3 does somethingraising suspicion, then even if all ten boats randomize theirpositions/locations the next day, the acoustic fingerprinting systemsand methods described herein can identify exactly boat #3. In oneembodiment, the acoustic fingerprint identification is so accurate thatit can prove in precise forensic detail which of a set of devices isconnected to an acoustic fingerprint. Thus, for example, the acousticfingerprint can be used by government organizations to positivelyidentify devices that were used in the performance of a suspicious orillicit activity while under passive surveillance.

In one embodiment, the acoustic fingerprints are compact representationsof the operational acoustic output (e.g., motor/engine noise) of adevice. The compact size of the acoustic fingerprint enables or enhancesreal-time identification of target devices. In one embodiment, theacoustic fingerprinting systems and methods transforming an originalacoustic wave form-which has a high sampling rate (for example 20kilohertz) that is too large to import into cloud systems due tobandwidth considerations-into a cluster of time series signals at alower sampling rate. This substantial reduction in size enablesimportation into a cloud environment along with other telemetryvariables, and due to the form of transformation, there is little if anyloss in accuracy of identification when using the acoustic fingerprintas shown and described herein.

These and other advantages are enabled by the acoustic fingerprintingsystems, methods, and other embodiments described herein.

—Cloud or Enterprise Embodiments—

In one embodiment, the acoustic fingerprinting system 100 is acomputing/data processing system including an application or collectionof distributed applications for enterprise organizations. Theapplications and computing system 100 may be configured to operate withor be implemented as a cloud-based networking system, aninfrastructure-as-a-service (IAAS), platform-as-a-service (PAAS), orsoftware-as-a-service (SAAS) architecture, or other type of networkedcomputing solution. In one embodiment the acoustic fingerprinting systemis a centralized server-side application that provides at least thefunctions disclosed herein and that is accessed by many users viacomputing devices/terminals communicating with the computing system 100(functioning as the server) over a computer network.

In one embodiment, one or more of the components described herein may inintercommunicate by electronic messages or signals. These electronicmessages or signals may be configured as calls to functions orprocedures that access the features or data of the component, such asfor example application programming interface (API) calls. In oneembodiment, these electronic messages or signals are sent between hostsin a format compatible with transmission control protocol/Internetprotocol (TCP/IP) or other computer networking protocol. In oneembodiment, components may (i) generate or compose an electronic messageor signal to issue a command or request to another component, (ii)transmit the message or signal to other components, and (iii) parse thecontent of an electronic message or signal received to identify commandsor requests that the component can perform, and in response toidentifying the command, the component will automatically perform thecommand or request.

In one embodiment, one or more of the components described herein areconfigured as program modules stored in a non-transitory computerreadable medium. The program modules are configured with storedinstructions that when executed by at least a processor cause thecomputing device to perform the corresponding function(s) as describedherein.

—Computing Device Embodiment—

FIG. 6 illustrates an example computing device 600 that is configuredand/or programmed as a special purpose computing device with one or moreof the example systems and methods described herein, and/or equivalents.The example computing device may be a computer 605 that includes atleast one hardware processor 610, a memory 615, and input/output ports620 operably connected by a bus 625. In one example, the computer 605may include acoustic fingerprinting logic 630 configured to facilitateacoustic fingerprint identification of devices, similar to logic,systems, and methods shown and described herein for example withreference to FIGS. 1-5 .

In different examples, the logic 630 may be implemented in hardware, anon-transitory computer-readable medium 637 with stored instructions,firmware, and/or combinations thereof. While the logic 630 isillustrated as a hardware component attached to the bus 625, it is to beappreciated that in other embodiments, the logic 630 could beimplemented in the processor 610, stored in memory 615, or stored indisk 635.

In one embodiment, logic 630 or the computer is a means (e.g.,structure: hardware, non-transitory computer-readable medium, firmware)for performing the actions described. In some embodiments, the computingdevice may be a server operating in a cloud computing system, a serverconfigured in a Software as a Service (SaaS) architecture, a smartphone, laptop, tablet computing device, and so on.

The means may be implemented, for example, as an ASIC programmed tofacilitate acoustic fingerprint identification of devices. The means mayalso be implemented as stored computer executable instructions that arepresented to computer 605 as data 640 that are temporarily stored inmemory 615 and then executed by processor 610.

Logic 630 may also provide means (e.g., hardware, non-transitorycomputer-readable medium that stores executable instructions, firmware)for performing acoustic fingerprint identification of devices.

Generally describing an example configuration of the computer 605, theprocessor 610 may be a variety of various processors including dualmicroprocessor and other multi-processor or multi-core architectures. Amemory 615 may include volatile memory and/or non-volatile memory.Non-volatile memory may include, for example, ROM, PROM, and so on.Volatile memory may include, for example, RAM, SRAM, DRAM, and so on.

A storage disk 635 may be operably connected to the computer 605 via,for example, an input/output (I/O) interface (e.g., card, device) 645and an input/output port 620 that are controlled by at least aninput/output (I/O) controller 647. The disk 635 may be, for example, amagnetic disk drive, a solid-state disk drive, a floppy disk drive, atape drive, a Zip drive, a flash memory card, a memory stick, and so on.Furthermore, the disk 635 may be a CD-ROM drive, a CD-R drive, a CD-RWdrive, a DVD ROM, and so on. The memory 615 can store a process 650and/or data 640, for example. The disk 635 and/or the memory 615 canstore an operating system that controls and allocates resources of thecomputer 605.

In one embodiment, non-transitory computer-readable medium 637 includescomputer-executable instructions such as software. In generalcomputer-executable instructions are designed to be executed by one ormore processors 610 accessing memory 615 or other components of computer605. These computer-executable instructions may include, for example,computer-executable code and source code that may be compiled intocomputer-executable code or interpreted for execution.

The computer 605 may interact with, control, and/or be controlled byinput/output (I/O) devices via the input/output (I/O) controller 647,the I/O interfaces 645, and the input/output ports 620. Input/outputdevices may include, for example, one or more displays 670, printers 672(such as inkjet, laser, or 3D printers), audio output devices 674 (suchas speakers or headphones), text input devices 680 (such as keyboards),cursor control devices 682 for pointing and selection inputs (such asmice, trackballs, touch screens, joysticks, pointing sticks, electronicstyluses, electronic pen tablets), audio input devices 684 (such asacoustic transducers as described in detail above, or external audioplayers), video input devices 686 (such as video and still cameras, orexternal video players), image scanners 688, video cards (not shown),disks 635, network devices 655, and so on. The input/output ports 620may include, for example, serial ports, parallel ports, and USB ports.

The computer 605 can operate in a network environment and thus may beconnected to the network devices 655 via the I/O interfaces 645, and/orthe I/O ports 620. Through the network devices 655, the computer 605 mayinteract with a network 660. Through the network, the computer 605 maybe logically connected to remote computers 665. Networks with which thecomputer 605 may interact include, but are not limited to, a LAN, a WAN,and other networks.

In one embodiment, computer 605 may be configured with hardware toprocess heavy workloads (such as those involved in acousticfingerprinting from fine-frequency acoustic output) at high speed withhigh reliability, for example by having high processing throughputand/or large memory or storage capacity. In one embodiment, computer 605is configured to execute cloud-scale applications locally where networkaccess is limited.

Definitions and Other Embodiments

In another embodiment, the described methods and/or their equivalentsmay be implemented with computer executable instructions. Thus, in oneembodiment, a non-transitory computer readable/storage medium isconfigured with stored computer executable instructions of analgorithm/executable application that when executed by a machine(s)cause the machine(s) (and/or associated components) to perform themethod. Example machines include but are not limited to a processor, acomputer, a server operating in a cloud computing system, a serverconfigured in a Software as a Service (SaaS) architecture, a smartphone, and so on). In one embodiment, a computing device is implementedwith one or more executable algorithms that are configured to performany of the disclosed methods.

In one or more embodiments, the disclosed methods or their equivalentsare performed by either: computer hardware configured to perform themethod; or computer instructions embodied in a module stored in anon-transitory computer-readable medium where the instructions areconfigured as an executable algorithm configured to perform the methodwhen executed by at least a processor of a computing device.

While for purposes of simplicity of explanation, the illustratedmethodologies in the figures are shown and described as a series ofblocks of an algorithm, it is to be appreciated that the methodologiesare not limited by the order of the blocks. Some blocks can occur indifferent orders and/or concurrently with other blocks from that shownand described. Moreover, less than all the illustrated blocks may beused to implement an example methodology. Blocks may be combined orseparated into multiple actions/components. Furthermore, additionaland/or alternative methodologies can employ additional actions that arenot illustrated in blocks. The methods described herein are limited tostatutory subject matter under 35 U.S.C. § 101.

The following includes definitions of selected terms employed herein.The definitions include various examples and/or forms of components thatfall within the scope of a term and that may be used for implementation.The examples are not intended to be limiting. Both singular and pluralforms of terms may be within the definitions.

References to “one embodiment”, “an embodiment”, “one example”, “anexample”, and so on, indicate that the embodiment(s) or example(s) sodescribed may include a particular feature, structure, characteristic,property, element, or limitation, but that not every embodiment orexample necessarily includes that particular feature, structure,characteristic, property, element or limitation. Furthermore, repeateduse of the phrase “in one embodiment” does not necessarily refer to thesame embodiment, though it may.

A “data structure”, as used herein, is an organization of data in acomputing system that is stored in a memory, a storage device, or othercomputerized system. A data structure may be any one of, for example, adata field, a data file, a data array, a data record, a database, a datatable, a graph, a tree, a linked list, and so on. A data structure maybe formed from and contain many other data structures (e.g., a databaseincludes many data records). Other examples of data structures arepossible as well, in accordance with other embodiments.

“Computer-readable medium” or “computer storage medium”, as used herein,refers to a non-transitory medium that stores instructions and/or dataconfigured to perform one or more of the disclosed functions whenexecuted. Data may function as instructions in some embodiments. Acomputer-readable medium may take forms, including, but not limited to,non-volatile media, and volatile media. Non-volatile media may include,for example, optical disks, magnetic disks, and so on. Volatile mediamay include, for example, semiconductor memories, dynamic memory, and soon. Common forms of a computer-readable medium may include, but are notlimited to, a floppy disk, a flexible disk, a hard disk, a magnetictape, other magnetic medium, an application specific integrated circuit(ASIC), a programmable logic device, a compact disk (CD), other opticalmedium, a random access memory (RAM), a read only memory (ROM), a memorychip or card, a memory stick, solid state storage device (SSD), flashdrive, and other media from which a computer, a processor or otherelectronic device can function with. Each type of media, if selected forimplementation in one embodiment, may include stored instructions of analgorithm configured to perform one or more of the disclosed and/orclaimed functions. Computer-readable media described herein are limitedto statutory subject matter under 35 U.S.C. § 101.

“Logic”, as used herein, represents a component that is implemented withcomputer or electrical hardware, a non-transitory medium with storedinstructions of an executable application or program module, and/orcombinations of these to perform any of the functions or actions asdisclosed herein, and/or to cause a function or action from anotherlogic, method, and/or system to be performed as disclosed herein.Equivalent logic may include firmware, a microprocessor programmed withan algorithm, a discrete logic (e.g., ASIC), at least one circuit, ananalog circuit, a digital circuit, a programmed logic device, a memorydevice containing instructions of an algorithm, and so on, any of whichmay be configured to perform one or more of the disclosed functions. Inone embodiment, logic may include one or more gates, combinations ofgates, or other circuit components configured to perform one or more ofthe disclosed functions. Where multiple logics are described, it may bepossible to incorporate the multiple logics into one logic. Similarly,where a single logic is described, it may be possible to distribute thatsingle logic between multiple logics. In one embodiment, one or more ofthese logics are corresponding structure associated with performing thedisclosed and/or claimed functions. Choice of which type of logic toimplement may be based on desired system conditions or specifications.For example, if greater speed is a consideration, then hardware would beselected to implement functions. If a lower cost is a consideration,then stored instructions/executable application would be selected toimplement the functions. Logic is limited to statutory subject matterunder 35 U.S.C. § 101.

An “operable connection”, or a connection by which entities are“operably connected”, is one in which signals, physical communications,and/or logical communications may be sent and/or received. An operableconnection may include a physical interface, an electrical interface,and/or a data interface. An operable connection may include differingcombinations of interfaces and/or connections sufficient to allowoperable control. For example, two entities can be operably connected tocommunicate signals to each other directly or through one or moreintermediate entities (e.g., processor, operating system, logic,non-transitory computer-readable medium). Logical and/or physicalcommunication channels can be used to create an operable connection.

“User”, as used herein, includes but is not limited to one or morepersons, computers or other devices, or combinations of these.

While the disclosed embodiments have been illustrated and described inconsiderable detail, it is not the intention to restrict or in any waylimit the scope of the appended claims to such detail. It is, of course,not possible to describe every conceivable combination of components ormethodologies for purposes of describing the various aspects of thesubject matter. Therefore, the disclosure is not limited to the specificdetails or the illustrative examples shown and described. Thus, thisdisclosure is intended to embrace alterations, modifications, andvariations that fall within the scope of the appended claims, whichsatisfy the statutory subject matter requirements of 35 U.S.C. § 101.

To the extent that the term “includes” or “including” is employed in thedetailed description or the claims, it is intended to be inclusive in amanner similar to the term “comprising” as that term is interpreted whenemployed as a transitional word in a claim.

To the extent that the term “or” is used in the detailed description orclaims (e.g., A or B) it is intended to mean “A or B or both”. When theapplicants intend to indicate “only A or B but not both” then the phrase“only A or B but not both” will be used. Thus, use of the term “or”herein is the inclusive, and not the exclusive use.

What is claimed is:
 1. A method, comprising: generating a targetacoustic fingerprint from acoustic output of a target device; generatinga similarity metric that quantifies similarity of the target acousticfingerprint to a reference acoustic fingerprint of a reference device;comparing the similarity metric to a threshold; in response to a firstcomparison result of the comparing of the similarity metric to thethreshold, indicating that the target device matches the referencedevice; and in response to a second comparison result of the comparingof the similarity metric to the threshold, indicating that the targetdevice does not match the reference device.
 2. The method of claim 1,further comprising: in response to the first comparison result,indicating that the target device is the reference device; and inresponse to the second comparison result, indicating that the targetdevice is not the reference device.
 3. The method of claim 1, furthercomprising: in response to the first comparison result, indicating thatthe target device is a same type of device as the reference device; andin response to the second comparison result, indicating that the targetdevice is a different type of device as the reference device.
 4. Themethod of claim 1, further comprising: measuring acoustic output of adevice; decomposing a spectrum of the measurements into a set offrequencies; partitioning the set of the frequencies into bins coveringranges of the set of the frequencies; selecting a set of one or more ofthe bins to be a basis of an acoustic fingerprint; and samplingrepresentative frequencies of the set of bins at intervals over a periodof time to produce a set of component time series signals; wherein (i)where the device is the target device, the target acoustic fingerprintis generated from the set of the component time series signals, and (ii)where the device is the reference device, the reference acousticfingerprint is generated from the set of the component time seriessignals.
 5. The method of claim 1, wherein the generating the similaritymetric further comprises: for a set of one or more target componentsignals included in the target acoustic fingerprint, determining a meanabsolute error between the target component signal and a correspondingreference component signal of the reference acoustic fingerprint; andfinding a sum of the mean absolute errors, wherein the similarity metricis the sum of the mean absolute errors.
 6. The method of claim 1,further comprising generating the reference acoustic fingerprint frommeasurements of acoustic output of the reference device.
 7. The methodof claim 1, further comprising retrieving the reference acousticfingerprint from a library of acoustic fingerprints, wherein thereference acoustic fingerprint is stored in the library in associationwith information describing the reference device.
 8. The method of claim1, further comprising passively recording the acoustic output of thetarget device with one or more acoustic transducers.
 9. The method ofclaim 1, further comprising: recording acoustic output of a plurality ofdevices including the target device with one or more acoustictransducers; and identifying the target device from among the pluralityof devices.
 10. The method of claim 1, wherein the target device is avehicle.
 11. The method of claim 10, wherein the vehicle is one of awatercraft or an aircraft.
 12. A non-transitory computer-readable mediumstoring computer-executable instructions that when executed by at leasta processor of a computer cause the computer to: generate a targetacoustic fingerprint from acoustic output of a target device; generate asimilarity metric that quantifies similarity of the target acousticfingerprint to a reference acoustic fingerprint of a reference device;comparing the similarity metric to a threshold; in response to a firstcomparison result of the comparing the similarity metric to thethreshold, indicating that the target device matches the referencedevice; and in response to a second comparison result of the comparingthe similarity metric to the threshold, indicating that the targetdevice does not match the reference device.
 13. The non-transitorycomputer-readable medium of claim 12, wherein the instructions furthercause the computer to: in response to the first comparison result,indicate that the target device is the reference device; and in responseto the second comparison result, indicate that the target device is notthe reference device.
 14. The non-transitory computer-readable medium ofclaim 12, wherein the instructions further cause the computer to: inresponse to the first comparison result, indicate that the target deviceis a same type of device as the reference device; and in response to thesecond comparison result, indicate that the target device is a differenttype of device as the reference device.
 15. The non-transitorycomputer-readable medium of claim 12, wherein the instructions furthercause the computer to: decompose a spectrum of measurements of acousticoutput of a device into a set of frequencies; partition the set offrequencies into bins covering ranges of the frequencies; select a setof one or more of the bins to be a basis of an acoustic fingerprint;sample representative frequencies of the set of bins at intervals over aperiod of time to produce a set of component time series signals; andcompensate for ambient noise in the component time series signals basedon values for the component time series signals predicted by a machinelearning algorithm; wherein (i) where the device is the target device,the target acoustic fingerprint is generated from the set of thecomponent time series signals, and (ii) where the device is thereference device, the reference acoustic fingerprint is generated fromthe set of the component time series signals.
 16. The non-transitorycomputer-readable medium of claim 12, wherein the instructions forgeneration of the similarity metric further cause the computer to: for aset of one or more reference component signals included in the targetacoustic fingerprint, determine a mean absolute error between thereference component signal and a corresponding component signal of thereference acoustic fingerprint; and find a sum of the mean absoluteerrors, wherein the similarity metric is the sum of the mean absoluteerrors.
 17. A computing system, comprising: a processor; a memoryoperably connected to the processor; a non-transitory computer-readablemedium operably connected to the processor and memory and storingcomputer-executable instructions that when executed by at least theprocessor cause the computing system to: generate a target acousticfingerprint from measurements of acoustic output of a target device;generate a similarity metric that quantifies similarity of the targetacoustic fingerprint to a reference acoustic fingerprint of a referencedevice; compare the similarity metric to a threshold; in response to afirst comparison result of the comparing the similarity metric to thethreshold, indicating that the target device matches the referencedevice; and in response to a second comparison result of the comparingthe similarity metric to the threshold, indicating that the targetdevice does not match the reference device.
 18. The computing system ofclaim 17, wherein the instructions further cause the computing systemto: in response to the first comparison result, indicate that the targetdevice is the reference device; and in response to the second comparisonresult, indicate that the target device is not the reference device. 19.The computing system of claim 17, further comprising one or moreacoustic transducers, wherein the instructions further cause thecomputing system to passively record the acoustic output of the targetdevice with the one or more acoustic transducers to generate themeasurements of the acoustic output.
 20. The computing system of claim17, further comprising one or more acoustic transducers, wherein theinstructions further cause the computing system to: passively recordacoustic output of a plurality of devices including the target devicewith the one or more acoustic transducers; and identify the targetdevice from among the plurality of devices.